Background

This document explores the skill of ECMWF’s seasonal forecast, to predict dry spells. ECMWF releases a forecast each month. This forecast includes projected total precipitation per month for 1 to 6 months ahead. The 1 month ahead is the month the forecast was released, and the release date is always the 13th of the month. I.e. the 1 month leadtime only becomes available when we are alread two weeks into that month.

ECMWF’s forecast is a probabilistic forecast, meaning it consists of several members (=models) each having their projected precipitation. This is what in this document is referred to as % of members, and can be interpreted as a probability of the event occurring.

Definitions

This analysis

  • Only looks at the Southern region since almost all historical dry spells occurred in that region
  • Only looks at the months of December, January, February since these months the crops are most sensitive to dry spells
  • Classifies a month as having experienced a dry spell, if at least 7 days of that month were within a dry spell and at least 3 adm2’s within the southern region experienced a dry spell
  • Computes the monthly precipitation as the mean value of all cells within the admin1

Percentage based

Set percentage and thereafter determine threshold

50% probability

Distribution of precipitation

The figure below shows the distribution of probabilities with and without a dry spell per leadtime. From this figure it can be seen that the two distributions are not very separable.

Misses and false alarms

Based on the misses and false alarms, the threshold is set to 220

Since we focus on leadtime 2 and 4, we only show these confusion matrices

Heatmaps

60% probability

Misses and false alarms

Based on the misses and false alarms, the threshold is set to 240

Since we focus on leadtime 2 and 4, we only show these confusion matrices

Heatmaps

Threshold based

Set threshold and thereafter determine percentage. In this case the percentage was set per lead time, where the percentage was chosen such that the number of months the forecast would trigger was closest to the number of months the observed precipitation was below the threshold. ### What is the percentage of ensemble members forecasted below a certain threshold? The figure below shows for how many months the % of ensemble members was below 170 mm. The 170 mm was the threshold set based on overlap of dry spells and monthly precipitation analysis.

We can see that the number of months lowers as the percentage increases, which is expected. While there are slight differences between lead times, the pattern is comparable.

To set the threshold of % of members, we look at the observed data and count how many months had <=170mm. This were 18 months. We then choose the % of members for each leadtime which has closest to 18 months forecasted <= 170mm

What is the miss and false alarm rate per leadtime to forecast below 170 mm monthly precipitation?

After setting the % of members, we get a list of forecasted months for which we would have triggered. Now we can compare these with the observed months with less than 170 mm of precipitation.

NOTE: false alarms is FP/(FP+TP)! i.e. the percentage of times the trigger was met but the monthly precipitation was not below 170

What is the miss and false alarm rate per leadtime to forecast dry spells?

NOTE: false alarms is FP/(FP+TP)! i.e. the percentage of times the trigger was met but there was no dry spell